Network anomaly detection based on Eigen equation compression.

Mining abnormal patterns from heterogeneous time%2Dseries with irrelevant features for fault event detection.
We address the issue of detecting fault events in multivariate time series. We suppose the following realistic situation: A) the features to which multivariate time series correspond are heterogeneous; B) relative to a large number of normal examples, only a small number of examples of fault events are available in advance; and C) many features irrelevant to fault events are included. In such a situation, we require real-time, high-accuracy processing. We propose an algorithm to resolve the issue. Key ideas in it include: 1) transforming the time-series for each feature into a sequence of anomaly scores, in order to map heterogeneous features to homogeneous features (an anomaly score indicates the degree of anomaly relative to an ordinal sequence) and then representing the pattern of a fault event in terms of anomaly score vectors; 2) selecting features specifying a fault event by means of iterative optimization using both normal and fault anomaly score vectors. We then monitor the degree of abnormal with regard to test anomaly score vectors by matching with the abnormal patterns. We demonstrate the effectiveness of our proposed algorithm through an application to an actual automobile fault diagnosis data set. 1
Latent variable mining with its applications to anomalous behavior detection.
In this paper, we propose a new approach to anomaly detection by looking at the latent variable space to make the first step toward latent anomaly detection. Most conventional approaches to anomaly detection are concerned with tracking data which are largely deviated from the ordinary pattern. In this paper, we are instead concerned with the issue of how to track changes happening in the latent variable space consisting of the meta information existing behind observed data. For example, in the case of masquerade detection, the conventional task was to detect anomalous command lines related to masqueraders ’ malicious behaviors. Meanwhile we rather attempt to track changes of behavior patterns such as writing mails, making software, etc. which are information of more abstract level than command lines. The key ideas of the methods are: 1) constructing the model variation vector, which is introduced relative to the latent variable space, and 2) the latent anomaly detection is reduced to the issue of change-point detection for the time series that the model variation vector forms. We demonstrate through the experimental results using artificial data set and UNIX command data set that our method has significantly enhanced the accuracy of existing anomaly detection methods.
Latent Variable Mining with Its Applications to Anomalous Behavior Detection.
In this paper, we propose a new approach to anomaly detection by looking at the latent variable space to make the first step toward latent anomaly detection. Most conventional approaches to anomaly detection are concerned with tracking data which are largely deviated from the ordinary pattern. In this paper, we are instead concerned with the issue of how to track changes happening in the latent variable space consisting of the meta information existing behind observed data. For example, in the case of masquerade detection, the conventional task was to detect anomalous command lines related to masqueraders ’ malicious behaviors. Meanwhile we rather attempt to track changes of behavior patterns such as writing mails, making software, etc. which are information of more abstract level than command lines. The key ideas of the methods are: 1) constructing the model variation vector, which is introduced relative to the latent variable space, and 2) the latent anomaly detection is reduced to the issue of change-point detection for the time series that the model variation vector forms. We demonstrate through the experimental results using artificial data set and UNIX command data set that our method has significantly enhanced the accuracy of existing anomaly detection methods.
Dynamic Model Selection With its Applications to Novelty Detection.

A Unifying Framework for Detecting Outliers and Change Points from Time Series.

Dynamic syslog mining for network failure monitoring.
Syslog monitoring technologies have recently received vast attentions in the areas of network management and network monitoring. They are used to address a wide range of important issues including network failure symptom detection and event correlation discovery. Syslogs are intrinsically dynamic in the sense that they form a time series and that their behavior may change over time. This paper proposes a new methodology of dynamic syslog mining in order to detect failure symptoms with higher confidence and to discover sequential alarm patterns among computer devices. The key ideas of dynamic syslog mining are 1) to represent syslog behavior using a mixture of Hidden Markov Models, 2) to adaptively learn the model using an on-line discounting learning algorithm in combination with dynamic selection of the optimal number of mixture components, and 3) to give anomaly scores using universal test statistics with a dynamically optimized threshold. Using real syslog data we demonstrate the validity of our methodology in the scenarios of failure symptom detection, emerging pattern identification, and correlation discovery.
Tracking dynamics of topic trends using a finite mixture model.

On%2DLine Unsupervised Outlier Detection Using Finite Mixtures with Discounting Learning Algorithms.
Outlier detection is a fundamental issue in data mining, speci cally in fraud detection, network intrusion detection, network monitoring, etc. SmartSifter, which we abbreviate as SS, is an outlier detection engine adrressing this problem from the viewpoint of statistical learning theory. This paper provides a theoretical basis for SS and empirically demonstrates its effectiveness. SS detects outliers in an online process through the on-line unsupervised learning of a probabilistic model (using a finite mixture model) of the information source. Each time a datum is input SS employs an on-line discounting learning algorithm to learn the probabilistic model. A score is given to the datum based on the learned model, with a high score indicating a high possibility of being a statistical outlier. The novel features of SS are: 1) it is adaptive to non-stationary sources of data; 2) a score has a clear statistical/information-theoretic meaning; 3) it is computationally inexpensive; and 4) it can handle both categorical and continuous variables. An experimental application to network intrusion detection shows that SS was able to identify data with high scores that corresponded to attacks, with low computational costs. Further experimental application has identified a number of meaningful rare cases in actual health insurance pathology data from Australia&#039;s Health Insurance Commission. 1.
Distributed cooperative mining for information consortia.

Topic analysis using a finite mixture model.
Addressed here is the issue of topic analysis&#039; which is used to determine a text&#039;s topic  structure, a representation indicating what topics are included in a text and how those topics  change within the text. Topic analysis consists of two main tasks: topic identification and  text segmentation to reflect topic changes. While topic analysis would be extremely useful in  a variety of text processing applications, no previous study has so far suificiently addressed it.
Mining product reputations on the Web.
Knowing the reputations of your own and/or competitors products is important for marketing and customer relationship management. It is, however, very costly to collect and analyze survey data manually. This paper presents a new framework for mining product reputations on the Internet. It automatically collects people&#039;s opinions about target products from Web pages, and uses text mining techniques to obtain reputations of the products. In advance, we generate, on the basis of human-tested examples, syntactic and linguistic rules to determine whether any given statement is an opinion or not, and the positive/negative nature of that opinion. We first collect statements regarding target products using a general search engine, then, using the rules, extract opinions from them and attach to each of the opinions the labels
A unifying framework for detecting outliers and change points from non%2Dstationary time series data.
We m&#039;e concerned vith the issues of outlier detection and change point detection from a data stream. In the area of data mining, there have been increased interest in these issues since the former is related to fraud detection, rare event discovery, etc., vhile the latter is related to event/trend change detection, activity monitoring, etc. Specifically, it is important to consider the situation where the data source is non-stationary, since the nature of data source may change over time in real applications. Although in most previous work outlier detection and change point detection have not been related explicitly, this paper presents a unifying frame- vork for dealing vith both of them on the basis of the theory of on-line learning of non-stationary time series. In this framevork a probabilistic model of the data source is inerementally learned using an on-line discounting learning algorithm, which can track the changing data source adaplively by forgetting the effect of past data gradually. Then the score for any given data is calculated to measure its deviation from the learned model, vith a higher score indicating a high possibility of being an outlier. Further change points in a data stream are detected by applying this scoring method into a time series of moving averaged losses for prediction using the learned model. Specifically ve develop an efficient algorithms for on-line discounting learning of auto-regression models from time series data, and demonstrate the validity of our framework through simulation and experimental applications to stock market data analysis.
Mining Open Answers in Questionnaire Data.

Text classification using ESC%2Dbased stochastic decision lists.
We propose a new method of text classification using stochastic decision lists. A stochastic decision list is an ordered sequence of IF-THEN rules, and our method can be viewed as a rule-based method for text classification having advantages of readability and refinability of acquired knowledge. Our method is unique in that decision lists are automatically constructed on the basis of the principle of minimizing Extended Stochastic Complexity (ESC), and with it we are able to construct decision lists that have fewer errors in classification. The accuracy of classification achieved with our method appears better than or comparable to those of existing rule-based methods. We have empirically demonstrated that rule-based methods like ours result in high classification accuracy when the categories to which texts are to be assigned are relatively specific ones and when the texts tend to be short. We have also empirically verified the advantages of rule-based methods over non-rule-based ones.
Discovering outlier filtering rules from unlabeled data combining a supervised learner with an unsupervised learner.

Mining from open answers in questionnaire data.

On%2Dline unsupervised outlier detection using finite mixtures with discounting learning algorithms.
Outlier detection is a fundamental issue in data mining, speci cally in fraud detection, network intrusion detection, network monitoring, etc. SmartSifter, which we abbreviate as SS, is an outlier detection engine adrressing this problem from the viewpoint of statistical learning theory. This paper provides a theoretical basis for SS and empirically demonstrates its effectiveness. SS detects outliers in an online process through the on-line unsupervised learning of a probabilistic model (using a finite mixture model) of the information source. Each time a datum is input SS employs an on-line discounting learning algorithm to learn the probabilistic model. A score is given to the datum based on the learned model, with a high score indicating a high possibility of being a statistical outlier. The novel features of SS are: 1) it is adaptive to non-stationary sources of data; 2) a score has a clear statistical/information-theoretic meaning; 3) it is computationally inexpensive; and 4) it can handle both categorical and continuous variables. An experimental application to network intrusion detection shows that SS was able to identify data with high scores that corresponded to attacks, with low computational costs. Further experimental application has identified a number of meaningful rare cases in actual health insurance pathology data from Australia&#039;s Health Insurance Commission. 1.
Extended Stochastic Complexity and Minimax Relative Loss Analysis.
. We are concerned with the problem of sequential prediction using a given hypothesis class of continuously-many prediction strategies. An effective performance measure is the minimax relative cumulative loss (RCL), which is the minimum of the worst-case difference between the cumulative loss for any prediction algorithm and that for the best assignment in a given hypothesis class. The purpose of this paper is to evaluate the minimax RCL for general continuous hypothesis classes under general losses. We first derive asymptotical upper and lower bounds on the minimax RCL to show that they match (k=2c) ln m within error of  o(ln m) where k is the dimension of parameters for the hypothesis class,  m is the sample size, and c is the constant depending on the loss function. We thereby show that the cumulative loss attaining the minimax RCL asymptotically coincides with the extended stochastic complexity (ESC), which is an extension of Rissanen&#039;s stochastic complexity (SC) into the decision-...
Text Classification Using ESC%2Dbased Stochastic Decision Lists.
We propose a new method of text classification using stochastic decision lists. A stochastic decision list is an ordered sequence of IF-THEN rules, and our method can be viewed as a rule-based method for text classification having advantages of readability and refinability of acquired knowledge. Our method is unique in that decision lists are automatically constructed on the basis of the principle of minimizing Extended Stochastic Complexity (ESC), and with it we are able to construct decision lists that have fewer errors in classification. The accuracy of classification achieved with our method appears better than or comparable to those of existing rule-based methods. We have empirically demonstrated that rule-based methods like ours result in high classification accuracy when the categories to which texts are to be assigned are relatively specific ones and when the texts tend to be short. We have also empirically verified the advantages of rule-based methods over non-rule-based ones.
Distributed Cooperative Bayesian Learning Strategies.

Minimax Relative Loss Analysis for Sequential Prediction Algorithms Using Parametric Hypotheses.

A Decision%2DTheoretic Extension of Stochastic Complexity and Its Applications to Learning.

Document Classification Using a Finite Mixture Model.
We propose a new method of classifying documents into categories. We define for each category a finite mixture model based on soft clustering of words. We treat the problem of classifying documents as that of conducting statistical hypothesis testing over finite mixture models, and employ the EM algorithm to efficiently estimate parameters in a finite mixture model. Exper- imental results indicate that our method outperforms existing methods.
Distributed Cooperative Bayesian Learning Strategies.

Document Classification Using a Finite Mixture Model
We propose a new method of classifying documents into categories. We define for each category a finite mixture model based on soft clustering of words. We treat the problem of classifying documents as that of conducting statistical hypothesis testing over finite mixture models, and employ the EM algorithm to efficiently estimate parameters in a finite mixture model. Exper- imental results indicate that our method outperforms existing methods.
On%2DLine Maximum Likelihood Prediction with Respect to General Loss Functions.

A Randomized Approximation of the MDL for Stochastic Models with Hidden Variables.

Randomized Approximate Aggregating Strategies and Their Applications to Prediction and Discrimination.

On%2Dline maximum likelihood prediction with respect to general loss functions.

alpha%2DHelix region prediction with stochastic rule learning.

A Loss Bound Model for On%2DLine Stochastic Prediction Algorithms

Probably Almost Discriminative Learning.

The Minimum L%2DComplexity Algorithm and its Applications to Learning Non%2DParametric Rules.

On Polynomial%2DTime Probably almost Discriminative Learnability.

Protein Secondary Structure Prediction Based on Stochastic%2DRule Learning.

Probably Almost Discriminative Learning.

A Learning Criterion for Stochastic Rules.
0. [WK91] S. Weiss and C. Kulikowski. Computer Systems that Learn. Morgan Kaufmann, San Mateo, CA, 1991. [Yam90] Kenji Yamanishi. A learning criterion for stochastic rules. In Proceedings of the 3nd Workshop on Computational Learning Theory, pages 67--81. published by Morgan Kaufmann, 1990. 70  [Pol84] David Pollard. Convergence of Stochastic Processes. Springer-Verlag, 1984. [Pol86] David Pollard. Rates of uniform almost-sure convergence for empirical processes indexed by unbounded classes of functions. manuscript, 1986. [Pol90] David Pollard. Empirical Processes: Theory and Applications, volume 2 of  NSF-CBMS Regional Conference Series in Probability and Statistics. Institute of Math. Stat. and Am. Stat. Assoc., 1990. [Qui89] Adolfo J. Quiroz. Metric entropy and learnability. Unpublished manuscript, Universidad Sim&#039;on Bol &#039; ivar, Caracas, Venezuela, 1989. [Ren70] A. Renyi. Probability Theory. N
A Loss Bound Model for On%2DLine Stochastic Prediction Strategies.

Learning Stochastic Motifs from Genetic Sequences.

